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Summary \ 

"Coach" can mean many things: 
five categories of literacy 
coaches in Reading First 



Simply knowing that literacy coaches are 
in schools does not imply anything about 
how those individuals spend their time — 
there is a difference between being a 
coach and doing coaching. 

Coaching has been heralded as an opportu- 
nity to provide professional development that 
is job-embedded, ongoing, directly related to 
the challenges teachers face in the classroom 
each day, and provided by people familiar with 
the context of the teachers’ work. Coaches — 
used in a variety of content areas but most 
commonly in literacy instruction — are often 
skilled teachers who step out of their class- 
rooms to help other teachers become more 
thoughtful and more effective in their instruc- 
tion. They work side-by-side with teachers in 
the classroom, observing, modeling, providing 
feedback, and planning lessons according to 
the needs and goals of individual teachers. At 
least in theory, this approach should address 
all the criteria of high-quality teacher profes- 
sional development. 

So tantalizing is the promise of coaching 
that in recent years states, districts, and 
schools across the nation, eager for a means to 



strengthen instruction and student learning, 
have rushed to implement literacy coaching 
(Russo, 2004). 

Because the expansion of coaching has oc- 
curred so quickly, federal, state, and local 
policymakers and practitioners who have little 
data about the effectiveness and impact of 
coaching must decide whether to use literacy 
coaches. Before the impact of coaching on 
student achievement can be demonstrated, 
however, educators need a clear picture of the 
qualifications and backgrounds of the people 
who become coaches and a description of what 
coaches actually do once they are in a coach- 
ing position. 

This report begins to develop this picture 
with data from and about coaches in Reading 
First — a federal project to improve reading 
outcomes for K-3 students in 5,200 low-per- 
forming elementary schools across the nation. 
In the five western states for which data were 
available (Alaska, Arizona, Montana, Wash- 
ington, and Wyoming), we found that coaches 
were mostly experienced teachers who were 
relatively inexperienced in the coaching role. 
We also found that the reality of how coaches 
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perform their jobs was more complex and 
varied than anticipated. In three of the five 
states, Reading First coaches were explicitly 
asked to spend 60 to 80 percent of their time in 
the classroom with teachers or working with 
teachers directly on their instruction. While 
coaches dedicated long hours to their jobs, 
they spent on average only 28 percent of their 
time working with teachers. This difference 
between the expectation of how coaches spend 
their time and the reality of their work has 
also appeared in other studies of coaching and 
is not unique to Reading First (Rollers, 2006; 
Bean and Zigmond, 2006; Knight, 2006). 

Although all coaches juggled multiple respon- 
sibilities and for the most part performed 
the same tasks, how they allocated their 
time across tasks and how they understood 
and described the focus of their work var- 
ied widely across individuals and settings. 
Utilizing both a cluster analysis of survey 
data and a qualitative analysis of interviews, 
we distinguished five categories of coaches: 
data-oriented, student-oriented, managerial, 
and two teacher-oriented categories— one that 



works largely with individual teachers and 
another that works with groups. It may be 
that other types of coaches also exist in other 
projects and settings. What we consider most 
important is that people who held the same 
job defined and performed their work in very 
different ways. 

Contrary to expectations, the prior educa- 
tion and experience of coaches did not predict 
which coach category they belonged to. Nor 
did school size predict how coaches performed 
their jobs. What our analyses did reveal, how- 
ever, was the significant relationship between 
the state in which a coach worked and the 
prevalence of particular coach categories. This 
relationship suggests that state guidance to 
Reading First schools and coaches contributed 
significantly to how coaches worked. Thus 
states, or any agency implementing educa- 
tional initiatives using coaches, have both a 
great deal of responsibility and a great op- 
portunity to influence what type of coach they 
employ to work in their schools and districts. 

June 2007 
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Simply knowing 
that literacy 
coaches are in 
schools does not 
imply anything 
about how those 
individuals are 
spending their 
time — there 
is a difference 
between being 
a coach and 
doing coaching. 



For years teachers have received their professional 
development primarily through one-shot work- 
shops provided by external trainers on topics that 
may or may not connect to teachers’ daily instruc- 
tional practices. Research has called this model 
into question, finding instead that professional 
development should be job-embedded, ongoing, 
directly related to the challenges teachers face in 
the classroom each day, and provided or supported 
by people familiar with the context of the teach- 
ers’ work (Guskey, 2000; Norton, 2001; Wood & 
McQuarrie, 1999). 

Coaching has been heralded as an opportunity to 
provide exactly this kind of professional devel- 
opment for teachers (Neufeld & Roper, 2003). 
Coaches— used in a variety of content areas but 
most commonly in literacy instruction — are often 



skilled teachers who step out of their classrooms 
to help other teachers become more thought- 
ful and more effective in their instruction. They 
work side-by-side with teachers in the classroom, 
observing, modeling, providing feedback, and 
planning lessons according to the needs and goals 
of individual teachers. At least in theory, this 
approach should address all the criteria of high- 
quality teacher professional development. 

So tantalizing is the promise of coaching that in 
recent years states, districts, and schools across the 
nation, eager for a means to strengthen instruc- 
tion and student learning, have rushed to imple- 
ment literacy coaching (Russo, 2004). Although 
coaching is not a new approach to professional 
development — variants of the coaching model 
date back to the 1930s (Hall, 2004)— the idea was 
reinvigorated by the frustration with traditional 
workshops and the need, under the No Child Left 
Behind Act of 2001, to find more effective means 
to enhance instruction and learning. Thus we see 
a rapid and widespread expansion of coaching, in- 
cluding in large urban districts (Russo, 2004) and 
in the entire state of Florida (Florida Department 
of Education, 2006). 

One of the largest initiatives using coaching has 
been Reading First, a federal project that aims to 
improve reading outcomes for students in low-per- 
forming K-3 schools. Reading First funded states, 
which in turn provided subgrants to districts and 
schools to adopt scientifically-based core reading 
programs, to utilize regular and standardized stu- 
dent assessments, and to provide interventions to 
struggling students. A large component of Read- 
ing First has been the professional development 
provided to teachers through institutes, work- 
shops, and most prominently site -based literacy 
coaches. Because Reading First mandated using a 
reading coach to provide professional development 
to teachers at grantee schools, more than 5,200 
schools have hired reading coaches (Moss, Jacob, 
Boulay, Horst, & Poulos, 2006). In addition, some 
districts with Reading First schools have extended 
parts of the model, including the use of a literacy 
coach, to other schools in their district (Center on 
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"COACH" CAN MEAN MANY THINGS 



Education Policy, 2006; Deussen, Autio, Nelses- 
tuen, Roccograndi, & Scott, 2006; Nelsestuen, 
Autio, & Smiley, 2006). 



2. How do coaches actually perform their jobs? 
That is, how do they spend their time, and 
what do they see as their focus? 



The upsurge in hiring coaches has prompted great 
professional interest in literacy coaching, and 
professional and trade journals are putting coach- 
ing center stage (Knight, 2006; Northwest Regional 
Educational Laboratory, 2006; Wren & Reed, 2005). 
In addition, the National Council of Teachers of 
English and the International Reading Association 
(IRA) recently established a set of standards for 
coaching (International Reading Association, 2004; 
National Council of Teachers of English, 2006) and 
a new clearinghouse for information about literacy 
coaching at the University of Colorado, Denver 
(National Council of Teachers of English, 2006). 



Because coaching has expanded so quickly, federal, 
state, and local policymakers and practitioners 
must decide whether to use literacy coaches with 
little data about what coaches do and whether 
coaching has an impact on student learning. These 
are issues that research needs to address. Before 
coaching can be linked to differences in student 
achievement, a clear picture of the qualifications 
and backgrounds of coaches and a description of 
what coaches actually do are needed. 



While coaches dedicated 
long hours to their jobs, 
they spent on average 
only 28 percent of their 
time with teachers 



Clarity about who coaches are 
and what they really do in schools 
will provide the context for 
understanding research on the 
relationship between professional 
development through teacher 
coaching and student outcomes. 
Articulating coaching roles based on implementa- 
tion provides researchers and stakeholders with an 
informed way to discuss coaching models. It also 
lays the foundation for bridging the gap between 
existing theory and implementation. 



Our research addresses two questions: 



1. Who becomes a reading coach? And what 
background, skills, and qualifications do 
coaches bring to their jobs? 



It answers these questions with data from and 
about Reading First coaches. The answers are 
relevant for Reading First— which includes about 
1,550 districts and 5,200 schools across the nation 
(Moss et al., 2006). They are also relevant for the 
many other schools and districts emulating Read- 
ing First, hiring literacy coaches, or wanting to 
draw lessons for their own literacy initiatives. 



SUMMARY OF FINDINGS AND 
THEIR IMPLICATIONS 

In answer to the first research question— who 
becomes a reading coach?— we found that in the 
five states for which we had data (Alaska, Arizona, 
Montana, Washington, and Wyoming), Read- 
ing First coaches were experienced teachers, but 
most had little or no prior coaching background. 
This matters to the states, districts, and schools 
that hire coaches, because it suggests that most 
new coaches need guidance, support, and specific 
training in coaching. 

More than half of the coaches had advanced 
degrees, 38 percent had advanced training 
specifically in literacy, and 22 percent held only 
bachelor’s degrees. This level of preparation is 
lower than that recently reported for Reading 
First coaches in Pennsylvania (Bean & Zigmond, 
2006), which found that 79 percent of Reading 
First coaches in that state held a masters degree 
or higher and 52 percent were certified as reading 
specialists. This suggests that when researchers 
study coaches, they may need to keep in mind re- 
gional variations in labor markets and educational 
background — our study covers the background 
and work of coaches in just five states and cannot 
be generalized to all other states. Regional varia- 
tions may also be important to policymakers and 
practitioners at the state and district levels when 
hiring, planning professional development, and 
providing technical assistance to coaches. In some 
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BOX 1 

Methodology 

The study data. We collected data 
from the 203 Reading First schools in 
five western states (Alaska, Arizona, 
Montana, Washington, and Wyo- 
ming). Surveys were administered 
in spring 2006 to K-3 teachers and 
literacy coaches at all 203 schools. 
Response rates were high: 75 to 88 
percent of teachers (depending on the 



state) and more than 90 percent of 
coaches returned their surveys. We 
had complete data from 190 coaches. 
Interviews were conducted with 
77 coaches and more than 300 K-3 
teachers at 77 Reading First schools, a 
subset of the 203 schools in the study 
(see the appendix for details on the 
data collection). 

The analyses. Surveys provided 
the information about coaches’ 



background and experience. We used 
the coaches’ answers to survey items 
that asked about the time they spent 
on various tasks to cluster individual 
coaches into five categories. The 
interviews provided more detailed 
descriptions and explanations of 
the five coach categories, explain- 
ing what coaches emphasized and 
defined as most important in their 
work. 



places, new literacy coaches may come in with 
more background in reading, while in other areas, 
this literacy expertise must be built on the job. 

Most coaches were hired from within their dis- 
tricts or schools. From our data, we were unable to 
investigate whether this was due to a preference for 
hiring from within or to low numbers of exter- 
nal applicants. Given the high standards that the 
International Reading Association (2004) has sug- 
gested that coaches should bring to their jobs, more 
attention to labor markets, coach qualifications, and 
recruitment and hiring practices is merited. 

In answering the second question — how do 
coaches perform their jobs?— we found that 
reality deviated sharply from the expectations of 
Reading First state project staff. In three of the 
five states studied, Reading First coaches were 
asked to spend 60 to 80 percent of their time in the 
classroom with teachers or working with teach- 
ers directly on their instruction. While coaches 
dedicated long hours to their jobs, they spent on 
average only 28 percent of their time with teach- 
ers. Even though this was dramatically lower 
than state expectations, it is a significant portion 
of time in comparison to what we know about 
coaches in other settings. For example, the Rollers’ 
(2006) study found that coaches spent, on average, 
two to four hours a week observing, demonstrat- 
ing lessons, and discussing the lessons taught and 
less than an hour planning lessons with teachers— 
for less than 15 percent of a 40 -hour workweek. 



While Bean and Zigmond’s study of Reading First 
coaches in Pennsylvania (2006) did not report 
percentage of time in the same way, it found that 
coaches spent more time on school improvement- 
related activities than on working with teachers 
in the classroom, and many coaches in the study 
complained that assessments and clerical tasks left 
them too little time to work with teachers. Outside 
Reading First, Knight (2006) found that coaches’ 
chief complaint about their job was that a long list 
of non-instructional tasks prevented them from 
working with teachers as often as they wanted. 

Our study documented the other tasks that filled 
the 72 percent of time that coaches did not spend 
directly with teachers. Working with data took up 
25 percent of coaches’ time— evenly spread among 
administering or overseeing assessments, manag- 
ing data, and interpreting data. Meetings, student 
interventions, documentation, and other tasks took 
up the rest of the time. For some coaches, this time 
spent on other tasks was the result of demands 
placed on them by the school, district, and state. But 
for others, it was a way for coaches to avoid coach- 
ing teachers because they did not feel comfortable 
or appropriately qualified to do so. Individuals and 
agencies responsible for hiring, training, and evalu- 
ating coaches could provide extra support to build 
confidence in coaches who need it. 

Although all coaches juggled multiple respon- 
sibilities and for the most part performed the 
same tasks, how they allocated their time across 
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People who hold the 
same job define and 
perform their work in 
very different ways 



tasks and how they understood 
and described their focus varied 
widely across individuals and 
settings. Utilizing both a cluster 
analysis and a qualitative analysis, 
we distinguished five categories of 
coaches: data-oriented, student-oriented, manage- 
rial, and two teacher-oriented categories, one that 
works largely with individual teachers and another 
that works with groups. 



• Data-oriented coaches spent on average 
45 percent of their workweek on data and 
assessment-related tasks. They described the 
focus of their work as facilitating the connec- 
tion between data and instruction. 



hire and train coaches. Training and technical as- 
sistance for Reading First coaches is provided pri- 
marily at the state level. How coaches performed 
in their jobs varied significantly by state. The 
state differences we detected in the distribution of 
coach categories suggest that state policy and the 
professional development and technical assistance 
provided by the state contributed to how coaches 
performed their jobs. This means that states, 
or any agency implementing reform initiatives 
using coaches, have both the responsibility and 
the opportunity to influence what type of coach 
they employ to work in their schools and districts. 
Future research might look more closely at the link 
between state-provided training and guidance and 
the way coaches perform their jobs. 



• Student- oriented coaches spent more time 
than other coaches working directly with stu- 
dents and the least (on average just 14 percent) 
working with teachers. They saw students as 
central to what they did. 

• Managerial coaches spent a substantial por- 
tion of their time keeping the systems run- 
ning in their schools — facilitating meetings 
and keeping up with paperwork. 

• Teacher- oriented coaches spent comparatively 
little time on paperwork and data-related 
tasks; they saw themselves primarily as pro- 
viders of professional development for teach- 
ers. They spent between 41 and 52 percent 

of their time working directly with teachers. 
Many of them worked with small groups of 
teachers, and about a third of them tended to 
work with individual teachers. 

Other types of coaches are possible in other popu- 
lations or in other programs, or the same catego- 
ries may show up with different frequencies in 
other settings. What we consider most important 
is that people who held the same job defined and 
performed their work in very different ways. 

The variation in how coaches performed on the job 
is a meaningful consideration for the people who 



Understanding the various ways the coach’s role is 
played out is important not only for the hiring and 
training agencies, but also for coaches themselves, 
who have demanding, time-consuming jobs. It 
provides them with a framework for self-assess- 
ment and can help them make decisions about 
how they allocate time. 

Although many lessons can be learned from Read- 
ing First schools, these schools are not necessarily 
typical elementary schools. Besides their particu- 
lar demographics (high poverty and history of low 
performance in reading), they have participated in 
a highly structured, well-funded federal program 
that is quite unlike other coaching initiatives in 
many ways. Programs that have less funding, 
part-time coaches, less professional development, 
or more school-level choice in curriculum or as- 
sessment may experience other patterns in how 
their coaches work. Nevertheless, we hope the 
descriptions in this report provide a starting point 
for thoughtful conversation about the job respon- 
sibilities, training, and day-to-day work of literacy 
coaches. 

A final important lesson from this study is a cau- 
tion for researchers not to assume that “coach” 
means only one thing— having a coach is not a 
uniform intervention. Simply knowing that liter- 
acy coaches are in schools does not imply anything 
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about how those individuals are spending their 
time, because there is a difference between being a 
coach and doing coaching. Researchers must define 
and measure what it is that coaches are doing in 
any study about coaching. This is particularly 
necessary as researchers investigate the question 
so many policymakers and practitioners raise: to 
what degree does investment in coaching make a 
difference in instruction and student achievement? 



directly opposed to the one-shot workshop model. 
They argue that effective professional development 
is experiential, grounded in inquiry and reflection, 
collaborative and interactive, connected to teach- 
ers’ work with students, sustained and intensive, 
and connected to other aspects of school change. 
Ideally, coaching meets all these criteria and dif- 
ferentiates support to match teachers’ previous 
training and experience. 



WHAT THE LITERATURE SAYS ABOUT 
COACHING— AND COACHES 

For all the attention to and interest in literacy 
coaching, it is still relatively new as an approach 
to large-scale professional development, and the 
research on its implementation and effective- 
ness is just emerging. In this review, we discuss 
what coaching is as an activity and what coaches 
(people in a job that includes coaching as well as 
other activities) actually do. We also review what 
is known about the impact of coaching on teachers 
and on student achievement. 



What is coaching? 

Coaching occurs when a more knowledgeable 
professional works closely with another profes- 
sional to increase productivity or to meet some 
predetermined outcome. In recent years, it has 
become a popular form of professional develop- 
ment in business and management; in the per- 
sonal realm, “life coaches” work with individuals 
to help them clarify and achieve individual goals. 
In education, literacy coaches support teachers 
in making instructional changes or decisions in 
order to improve student achievement in reading 
and writing. 

Literacy coaching is a response to criticism of 
more traditional professional development for 
teachers, which is often delivered by expert pre- 
senters over short periods of time with little or no 
follow-up. In fact, the key components of effective 
professional development for teachers described 
by Darling-Hammond and McLaughlin (1996) are 



Within this framework, there is debate about the 
degree to which coaching should be directive, 
telling teachers how to adjust their instruction, 
or collegial and targeted at enhancing teacher 
capacity for self-reflection (Costa & Garmston, 
2002; Joyce & Showers, 2002). In the directive 
model, the coach plays the role of an expert, 
identifying a teacher’s specific area of weakness 
or helping teachers implement a program with 
specific practices. In some instances, the job of a 
directive coach may be to ensure program fidelity. 
Some research suggests 
that teachers may be less 
likely to change in direc- 
tive approaches, although 
newer teachers may be 
more open to this type 
of coaching (Gersten, 

Morvant, & Brengelman, 

1995). 



Literacy coaches 
support teachers in 
making instructional 
changes or decisions 
in order to improve 
student achievement 
in reading and writing 



In contrast, coaching for self-reflection is a more 
collaborative model in which the coach and the 
teacher together raise questions about the effec- 
tiveness of instruction and make decisions about 
changes. Reflective coaching often includes help- 
ing teachers deepen their understanding about 
how students learn to read and write as well as 
about the teacher’s role in making learning effec- 
tive (Duncan, 2006; Toll, 2005). Cognitive coach- 
ing is a well-known example of reflective coaching 
(Costa & Garmston, 2002). 



What do coaches do besides coaching? 

Just as teaching is only one of many activities that 
teachers engage in, coaching is just one part of a 
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coach’s job. While teachers might, for example, 
plan lessons, grade papers, and take recess duty in 
addition to teaching children, coaches take on a 
wide range of activities: 

• Assisting teachers in implementing new cur- 
ricular programs (Poglinco et al., 2003) 

• Consulting with and mentoring teachers 
(Costa & Garmston, 2002) 

• Supporting teachers as they “apply knowledge, 
develop skills, polish technique and deepen 
their understanding” (Lyons & Pinnell, 2001) 

• Planning and conducting research and writ- 
ing grants (Walpole & McKenna, 2004) 

• Leading discussion groups (Sweeney, 2003) or 
study or book groups (Walpole & McKenna, 
2004) 

Not only do coaches have many responsibilities, 
but the term “coach” is used to describe many dif- 
ferent configurations: full-time coaches assigned 
to a single building, full-time coaches responsible 
for two or more buildings, part-time coaches (es- 
pecially in small schools), and teachers who pro- 
vide part-time peer coaching to their colleagues. 
While these positions, of course, have much in 
common, their differences are often disregarded in 
the literature, making it more difficult to interpret 
findings about implementation success and impact 
on both teachers and students. 



How might coaching affect teachers? 



While evidence indicates 
coaching is popular with 
teachers, evidence about 
coaching's impact on 
teacher behavior is mixed 



There is evidence that coaching is popular with 
teachers (Schwartz & McCarthy, 2003) and that 
teachers believe it improves their instruction and 
willingness to try new ap- 
proaches (Munro & Elliott, 
1987; Sparks & Bruder, 1987). 
Evidence about coaching’s 
impact on teacher behav- 
ior, however, is mixed. On 
the one hand, a number of 



studies suggest that coaching can change teacher 
practice. For example, Showers and Joyce (1996) 
found that teachers receiving coaching were 
more likely to use new strategies and to use them 
appropriately than were teachers receiving more 
traditional professional development. Neufeld and 
Roper (2003) reported that teachers whose profes- 
sional development includes coaching were more 
likely to try out the new ideas they learned. In a 
study in Topeka, Kansas, researchers found that 
within six weeks of the start of the school year, 85 
percent of teachers who worked with a coach had 
already implemented at least one teaching practice 
they had learned in a summer workshop, com- 
pared with just 10 percent of teachers who did not 
work with a coach (Knight, 2004). In a small study 
of primary grade teachers, Kohler, Crilley, Shearer, 
and Good (1997) found that teachers instituted 
and sustained more changes in their instructional 
approach when they worked with a peer coach. 

Yet other studies raise questions about the efficacy 
of coaching. In a study in the Netherlands, Veen- 
man, Denessen, Gerrits, and Kenter (2001) found 
that teachers who had been coached expressed 
higher levels of confidence in their teaching, but 
were not rated as more effective than teachers 
who had not received coaching. A small study by 
Gutierrez, Crosland, and Berlin (2001) concluded 
that coaching experiences did not help teachers 
fundamentally change their work in the class- 
room. This study, based on a review of surveys, 
interview data, and video-taped instruction, found 
that coaching focused on transmitting particular 
strategies often did not help teachers understand 
when or how to choose one instructional strategy 
rather than another. Additional studies, employ- 
ing rigorous methodologies, will be needed to help 
make sense of some of these equivocal findings. 

Despite questions about the impact of coaching 
on instructional practice, coaching does appear 
to have a positive impact on teacher collabora- 
tion (Schwartz & McCarthy, 2003). Enhanced 
collaboration is important because seeing how 
other teachers work helps teachers gauge their 
own skills, promotes reflection on how they can 
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improve (Stigler & Hiebert, 1999), and may foster 
critical inquiry into the connection between teach- 
ing practice and student outcomes (Lieberman, 
1995; Lord, 1994). But the same researchers who 
point out the importance of collaboration also 
note that it requires a great investment of time 
on a regular basis. In practice, shortage of time 
is cited as one of the most pressing challenges for 
coaches— insufficient time for coaches to attend to 
all teachers and for teachers to meet with coaches 
to confer about observations (Bean & Zigmond, 
2006; Knight, 2006; Schwartz & McCarthy, 2003). 



How might coaching affect student achievement? 

Proponents of coaching argue that there is good 
reason to expect that well-implemented coaching 
can improve instruction and student achievement. 
Based on the findings that teachers are more likely 
to adopt new practices if they have been coached 
(Knight, 2004; Neufeld & Roper, 2003; Showers & 
Joyce, 1996), advocates argue that coaching, prop- 
erly implemented, will alter and deepen teacher 
practice (Darling-Hammond, 2000). 

While some data suggest a potential connection 
between coaching and increased student achieve- 
ment, there are to date no rigorous scientific data 
that definitively demonstrate it (Greene, 2004; 
Poglinco et al., 2003). A number of evaluations and 
self-reports do posit a link between coaching and 
student outcomes, such as Nortons (2001) docu- 
mentation of the positive results of the statewide 
Alabama Reading Initiative (ARI) on the literacy 
of seventh-grade students. Also, researchers from 
the Foundation for California Early Literacy 
Learning (Schwartz & McCarthy, 2003) have 
reported that teachers perceived a positive effect 
on student achievement linked to coaching. Like- 
wise, Lyons and Pinnell (2001) saw a connection 
between literacy coaching and increased achieve- 
ment in reading and writing. None of these stud- 
ies, however, employed rigorous methodologies 
that could establish a causal link between coach- 
ing and student achievement. Future studies, such 
as the Institute of Education Sciences (2006) study 
on the impact of coaching on teacher practice 



and student achievement in early reading, will be 
needed to determine the effectiveness of coaching 
on student outcomes. 



Literacy coaching under Reading First 



In response to coachings promise of targeted, 
ongoing professional development for teachers, 
Reading First mandated the use of reading coaches 
at grantee schools. Other Reading First require- 
ments include schoolwide adoption of a core read- 
ing program and common reading assessments, 
the use of assessment data to make instructional 
decisions, and interventions for students at risk of 
reading difficulties. These components— and the 
very presence of mandated structures— provide 
Reading First coaches both support and challenges 
that may be somewhat different from those of 
coaches in other settings. 



While some data 
suggest a potential 
connection between 
coaching and increased 
student achievement, 
there are to date no 
rigorous scientific 
data that definitively 
demonstrate it 



About two years into the 
implementation of Read- 
ing First at the school 
level, the IRA (2004) 
released guidelines on the 
role and qualifications of 
the reading coach, includ- 
ing recommendations to 
policymakers, adminis- 
trators, reading special- 
ists, and coaches. The 

IRA accepts Doles (2004) definition of the coach 
as someone who “supports teachers in their daily 
work,” both informally (engaging in conversations, 
participating in study groups) and more formally 
(modeling, co-teaching lessons, conducting lesson 
studies with teachers). In order to be effective, 
coaches should a) be excellent teachers at the level 
they are coaching; b) have in-depth knowledge 
about reading processes, acquisition, instruction, 
and assessment (the kind of knowledge that comes 
from completion of a masters degree, reading 
certification, or intensive, yearlong training for 
new coaches); c) have experience working with 
teachers to improve their practice; d) be excellent 
presenters and familiar with presenting to teacher 
conferences and leading teacher groups; and e) 




8 



"COACH" CAN MEAN MANY THINGS 



have experience or preparation for observing and 
modeling in a classroom and providing construc- 
tive feedback to teachers. The IRA recommends 
that only teachers meeting all five of these criteria 
be hired as reading coaches. 



Teachers generally find 
demonstration lessons 
given by their coach 
to be useful but, in 
some places, complain 
that this help occurs 
far too infrequently 



These are demanding standards 
that may be hard for projects to 
meet when hiring new coaches. 
Even if the IRA guidelines had 
been available at the outset of 
Reading First, the standard might 
have proved particularly chal- 
lenging for a project that greatly 
increased the nationwide demand 
for reading coaches at the elementary level. These 
hiring challenges may be particularly acute in 
communities with smaller pools of qualified appli- 
cants or less access to advanced training opportu- 
nities. Empirical research is needed to document 
the preparation, assigned roles, and actual use of 
time on the job for the thousands of individuals 
hired as Reading First coaches in the past three 
years. 



Empirical data on Reading First are beginning 
to be released. The interim evaluation report on 
the implementation of Reading First (Moss et al., 
2006) provides the best overall national picture 
of Reading First coaching to date. It notes, for 
example, that more than a third of schools (38 
percent) are served by coaches who work less than 
full time, and 11 percent of coaches serve more 
than one school. Most coaches reported that the 
following activities were “absolutely central to 
their work”: providing professional development 
(94 percent), coaching school staff (91 percent), 
organizing professional development for K-3 
teachers (87 percent), compiling reading assess- 
ment data (88 percent), and administering and 
coordinating reading assessments (87 percent). 

The report also summarizes coaches’ ratings of 
the importance of various activities— more than 
75 percent of coaches rated nine of the 10 activi- 
ties asked about as “absolutely central.” What the 
report does not explore is how coaches balance the 
demands of these “absolutely central” priorities 



nor does it examine whether real time and effort 
correspond to coaches’ perceptions of what is 
important. 

Initial findings from non-representative national 
samples and from particular locales are beginning 
to be released (Bean & Zigmond, 2006; Roller, 
2006). These findings suggest that the educational 
and specific literacy preparation of coaches vary 
widely. They also note that coaches spend com- 
paratively small percentages of their time observ- 
ing teachers and conferring with them about their 
instruction. Teachers generally find demonstration 
lessons given by their coach to be useful but, in 
some places, complain that this help occurs far 
too infrequently and that coaches do not return 
to classrooms to fulfill promises of help (Alver- 
mann, Commeyras, Cramer, & Harnish, 2005); 
in some places, a third of teachers reported that 
they “never” worked with their coach or just “once 
or a few times a year” (Deussen et al., 2006). On 
the other hand, there are places where 75 percent 
of teachers report working with their coaches at 
least once a month (Autio, Nelsestuen, & Deussen, 
2006). 

Our study looks more closely at Reading First 
coaches in five western states: Alaska, Arizona, 
Montana, Washington, and Wyoming. These states 
were awarded Reading First grants between Au- 
gust 2002 and September 2003. Implementation in 
the first cohort of schools, and thus Reading First 
coaching, began either during the 2003/04 school 
year or in the summer of 2004. Arizona, Montana, 
and Washington brought on an additional cohort 
of schools in subsequent years. 

As coaches began their work in Reading First 
schools, they often found that their roles were only 
vaguely described, especially among those in the 
first cohort of schools. For example, in the first 
year (2003/04), some Arizona coaches complained 
of not receiving a job description until near the 
end of the school year. And at the end of the first 
year of implementation in Alaska Reading First 
schools, only 36 percent of coaches agreed that 
“my role as coach is clearly defined” (Smiley, 
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2006). Expectations became clearer over time, 
however. As new cohorts of schools joined the pro- 
gram in the second and third years of implementa- 
tion, more explicit responsibilities and expecta- 
tions were laid out. New Arizona coaches reported 
seeing their job description when they were hired. 
By the end of the second year of implementation in 
Alaska, 85 percent of coaches agreed that their role 
was clearly defined (Smiley, 2006). 

Training and guidance for new coaches was pro- 
vided primarily at the state level, and sometimes 
at the district level. Initially, state and district 
agencies had to design much of this training and 
guidance themselves, often with little on which to 
base their decisions. Four of the five states in this 
study began implementing Reading First before 
the IRA had established guidelines for the hiring 
of coaches. The National Center for Reading First 
Technical Assistance released a binder for coaches 
entitled “Leading for Reading Success: An Intro- 
ductory Guide for Reading First Coaches” in early 
2005, but that was nearly two years after most 
coaches had been hired in these states. Before 2005 
Reading First coaches pieced together the best 
guidance they could, learning from one another 
and often, as they said, “building the airplane 
while flying it.” 

Training for coaches in these five states took place 
at annual multi-day summer institutes, regular 
monthly or bimonthly statewide videoconferences, 
and ongoing building-level visits from a technical 
assistance provider. Additional guidance came in 
the form of state documents or monitoring instru- 
ments, such as implementation checklists (used in 
Arizona), school reading improvement plans (used 
in Montana), or bimonthly to-do lists (provided in 
Washington during the first year). 

State guidance to coaches had certain commonali- 
ties across states. For example, coaches in different 
states often received a portion of their training 
from the same external consultants (for example, 
Anita Archer’s workshops on student engagement 
or Jan Hasbroucks training for coaches), and they 
reported reading many of the same books, such as 



Overcoming Dyslexia (Shaywitz, 2003) or Bringing 
Words to Life (Beck, McKeown, & Lucas, 2002). 



At the same time, there were notable differences 
in what states asked coaches to do. For example, 
some states expected that coaches spend a spe- 
cific percentage of time in the classroom working 
with teachers, sometimes as high as 60 percent 
(Montana and Wyoming) to 80 percent of their 
time (Arizona). Other states focused on defin- 
ing the data component of the position (Alaska 
and, beginning in the second year, Washington), 
emphasizing the development of coach capacity to 
manipulate and understand reading assessment 
data as a vehicle for targeting student needs and 
changing teacher practice. In contrast, Arizona 
funded an assessment coordinator position in 
most schools to take the lead in assessment use 
in the school, greatly reducing expectations for 
coaches’ involvement in assessment. 



Before 2005 Reading 
First coaches pieced 
together the best 
guidance they could, 
learning from one 
another and often, as 
they said, "building the 
airplane while flying it" 



In order to ensure that 
grantee schools were 
implementing Reading 
First as planned, some 
states asked schools 
to provide substantial 
documentation of their 
activities — sometimes 
this work was assigned 
to coaches. In Montana, 

coaches were asked to compile binders of the year’s 
activities and complete implementation check- 
lists and to submit notes from study groups. In 
Arizona, schools were asked to provide detailed 
documentation of reading meetings (attendance, 
agendas, and notes), and in some cases the record 
keeping fell to the coach. These record-keeping 
requirements were probably more demanding 
than what literacy coaches in non-Reading First 
settings typically encounter. 



When individual Reading First coaches were 
hired, they were usually the school’s first coach. 
In the subsequent two to three years, as coaches 
balanced external expectations with their own 
skills and the demands of the local context, 
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they essentially defined the term “coach” for 
themselves and for the teachers with whom they 
worked. 



WHAT OUR RESEARCH FINDS 

Building on the findings of previous literature, our 
study looks at who becomes a Reading First coach 
and how coaches actually spend their time. 



Who becomes a Reading First coach? 

Demographic data obtained through the surveys 
provided a picture of coaches’ background and 
experience (table 1). In general, reading coaches 
were experienced teachers who brought little or 
no previous coaching experience to their position; 
in fact, more than two-thirds (70 percent) had no 
coaching experience prior to Reading First. 1 Many 
of the 30 percent of coaches with prior coaching 
experience, usually between one and three years, 
had only ever coached in their current school — 
not surprising, given that the widespread use of 
coaches is a recent phenomenon. 



On the other hand, coaches brought substantial 
teaching experience to their position, an average 
of 17 years. Many had 20 or more years of experi- 
ence, and it was rare that coaches had fewer than 
five years of teaching experience. 



Some states expected 
coaches to spend a 
specific percentage of 
time in the classroom 
working with teachers, 
while others focused 
on defining the 
data component 
of the position 



In terms of educational background, 38 percent 
had advanced training in literacy (a reading 
certification or a masters’ degree in reading). Half 
(54 percent) had a master’s degree in an area other 
than reading; some had this as 
well as advanced training in lit- 
eracy. About one-fifth (22 percent) 
held only a bachelor’s degree. 



While some reading coaches were 
hired externally, from somewhere 
as close as a neighboring school 
or as distant as another state, 
most (61 percent) were hired from 
within the school. These internal 



hires had worked at their current school for an 
average of 10 years. Reading coaches in urban lo- 
cations were more likely to be an external hire and 
new to their building. In rural areas, where the 
pool of qualified applicants may have been smaller, 
it was more common for coaches to be experienced 
teachers hired from within. 

Almost all coaches (93 percent) reported work- 
ing full-time in their positions. A few (11 percent) 
reported that there was another literacy coach in 
their building who also worked with K-3 reading 
teachers. 



What do Reading First coaches actually do? 

On the spring surveys, most coaches reported that 
they tended to work long hours— 49 hours a week, 
on average for full-time coaches; some reported 
routinely working 60-70 hours a week. Coaches 
also reported the average number of hours a week 
they spent on distinct tasks that comprise the 
bulk of their job responsibilities. To standardize 
across respondents, who might work very different 
numbers of hours a week, these responses were 
converted to percentage of time spent on each 
task a week. These findings, averaged across the 
190 coaches with complete data, are presented in 
table 2. 

On average, coaches spent 26 percent of their 
workweek actually coaching K-3 teachers: observ- 
ing K-3 teachers, providing feedback, demonstrat- 
ing lessons, or training groups of teachers (often at 
grade-level and staff meetings). Although Reading 
First coaches were supposed to focus on K-3 teach- 
ers, some reported coaching teachers in grades 4-6 
as well. When this time was included, the percent- 
age of time devoted to coaching increased to 28 
percent. These figures indicate that state expecta- 
tions that coaches should spend 60-80 percent of 
their time working directly with teachers were far 
removed from the reality of most coaches’ work. 

The table also shows that data- and assessment-re- 
lated work consumed, on average, the same propor- 
tion of the workweek as direct coaching activities: 
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TABLE 1 

Reading coach demographics 





N 


■■ 


Max 


Mean 


SD 


Years prior coaching experience 


184 


0 


10 


0.79 


1.58 


Years prior teaching experience 


185 


1 


42 


16.69 


9.20 


Years prior experience at current 
school (in any capacity) 


185 


1 


40 


8.32 


7.99 








N 


Percentage of coaches 


With prior coaching experience 






184 


30 




Without prior coaching experience 






184 


70 




Working full-time 






189 


91 




Working part-time 






189 


7 




Working with another K-3 literacy coach in current school 




184 


11 




With five or fewer years prior teaching experience 




185 


11 




With bachelor's degree only 3 






168 


22 




With advanced training in literacy 






168 


38 




With master's degree in area other than reading 




168 


54 




Hired from within 






185 


61 




Hired externally 






185 


34 




a. Arizona coaches did not provide information on their educational background on the 2006 surveys. For the educational background variables, only data 
from four states were included. 


TABLE 2 

Mean percentage of time spent on tasks, all coaches (N = 


190) 








Percentage of time 


One-on-one coaching (K-3) 








21 




Group coaching (K-3) 








5 




Coaching out-of-grade 








2 




Subtotal : Coaching 








28 




Administering/coordinating assessments 








8 




Managing data (entering, charting) 








8 




Using/interpreting data 








9 




Subtotal: Data & assessment 








25 




Planning interventions 








6 




Providing interventions directly 








4 




Subtotal: Interventions 








10 




Planning for/attending meetings 








14 




Attending professional development 








5 




Paperwork 








11 




Unrelated (subbing, bus duty, etc.) 








6 




Subtotal: Other 








36 




another 25 percent of their time. Data-related work 
included administering or coordinating student 




assessments, entering data, creating charts, and 
reviewing reading assessment data. 
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TABLE 3 

Mean percentage of time on activities and between group differences by coach category 





Cluster 1 


Cluster 2 


Cluster 3 


Cluster 4 


Cluster 5 


ANOVA 












Teacher- 


Teacher- 










Data- 


Student- 




oriented 


oriented 










oriented 


oriented 


Managerial 


(group) 


(individual) 


F( 4, 185)p 


Post-hoc test 


One-on-one coaching 


14 


10 


19 


25 


48 


80.13 


.000 


3>-|*** s> 2 *** 


(K— 3) 
















5>3*** 

4 > 1*** 4>2*** 

4>3*, 3>2*** 


Group coaching (K— 3) 


3 


4 


5 


10 


3 


25.08 


.000 


4 > 1*** 4>2*** 

4>3*** 4 ^*** 


Coaching out-of-grade 


1 


2 


2 


6 


1 


7.34 


.000 


4 > 1*** 4>2** 
4>3**, 4>5** 


Subtotal: Coaching 


18 


16 


25 


41 


52 


88.04 


.000 


3^1*** 5>2*** 
5>3*** 3 > 4*** 

4 > 1*** 4>2*** 

4>3***, 3>1*, 
3>2*** 


Administering/ 


13 


11 


7 


7 


3 


13.96 


.000 


1^*** ^ 4 *** 


coordinating assessments 
















1>5***,2>3**, 
2 >4**, 2>5*** 


Managing data (entering, 


16 


8 


6 


7 


3 


26.67 


.000 


1 > 2 *** i>3*** 


charting) 
















^ 4 *** ^ 3 *** 

2 >5**, 4>5* 


Using/interpreting data 


16 


8 


10 


8 


5 


23.25 


.000 


1 >2*** 1 >3*** 

^ 4 *** ^ 3 *** 

3>5***, 4>5* 


Subtotal: Data & assessment 


45 


27 


23 


23 


11 


52.27 


.000 


1 >2*** 1 >3*** 

^ 4 *** ^ 3 *** 
2>5*** 3>s*** 
4>3*** 


Planning interventions 


8 


6 


5 


6 


4 


2.87 


0.24 


1>5* 


Providing interventions 


2 


12 


2 


2 


3 


32.06 


.000 


2 >*|*** 2>3*** 


directly 
















2>4*** 2>5*** 


Subtotal: Interventions 


10 


18 


7 


8 


7 


19.05 


.000 


2>1**,2>3***, 
2>4*** 2>5*** 


Planning for/attending 


12 


12 


18 


14 


10 


10.49 


.000 


3 > 1 *** 3>2*** 


meetings 
















3 >4*, 3>5*** 


Attending professional 
development 


3 


4 


5 


5 


3 


3.52 


.008 




Paperwork 


10 


10 


17 


6 


12 


23.59 


.000 


3 > 1*** 3>2*** 
3>4***, 3>5 * 

2 >4**, 5>4** 


Unrelated (subbing, bus 


3 


12 


5 


4 


4 


11.70 


.000 


2 >'|*** 2>3*** 


duty, etc.) 
















2 > 4 ***, 2 >5** 


Subtotal: Other tasks 


28 


38 


45 


29 


29 








Number of coaches 


28 


45 


55 


41 


21 








Percentage of coaches 
(N = 190) 


15% 


24% 


29% 


21 % 


11 % 









* p < .05 ** p < .01 *** p < .001 
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Planning for and attending meetings took up the 
next largest block of coaches’ time (14 percent). 
This was closely followed by paperwork (11 per- 
cent) and interventions (10 percent), both plan- 
ning for them and, in some cases, delivering them 
directly. 

These data show that coaches held multifaceted 
positions with a complex array of responsibilities. 
They also show that coaches spent almost as much 
time, on average, on data-related activities as they 
did actually coaching teachers. But what the over- 
all averages obscure is the very large variation in 
responses across coaches. Some coaches said they 
spent no time on assessment-related work; others 
spent as much as 50 percent of their time on data 
and assessment. The overall averages also fail to 
make clear the common patterns in use of time 
among subgroups of coaches. 

To more thoroughly address the research ques- 
tion of how coaches actually spend their time on 
the job, a cluster analysis was conducted to create 
different categories of coaches based upon the 
percentage of time they spent on particular tasks. 
These results are shown in table 3, which presents 
the five coach categories, the mean percentage of 
time that individuals within each category spent 
on particular aspects of their position, and be- 
tween group differences across the categories. 

Data-oriented coaches. We called the first coach 
category “data-oriented” due to coaches’ focus on 
data and assessment tasks; almost half their work- 
week (45 percent) was spent on such responsibili- 
ties, including administration and coordination of 
assessment, data management, and use/interpreta- 
tion of data. This was significantly greater than all 
other clusters (p < .001). In comparison, data-ori- 
ented coaches spent less than half as much of their 
workweek directly coaching teachers (18 percent). 
As the last line in table 3 indicates, 15 percent of 
coaches in our study fell into this category. 

Student- oriented coaches. The second category 
of “student-oriented” coaches spend a dispro- 
portionate share of time providing interventions 



directly to students (12 percent, four to six times 
as much as the other categories). This was signifi- 
cantly greater than all other categories (p < .001). 
Although the average proportion of time that 
these coaches spent working directly with teachers 
(16 percent) was similar to that of data-oriented 
coaches, they were distinguished by not sharing 
the first cluster’s time on data-related tasks. Fur- 
thermore, the qualitative data (discussed below) 
revealed that these coaches’ focus, even when in 
the classroom, was on the activities of students , 
rather than teachers. Twenty-four percent of 
coaches fell into this category. 

Managerial coaches. We referred to coaches in the 
third coach category as “managerial” because of 
the disproportionate share of time they spent on 
paperwork and meetings. This stood out as practi- 
cally as well as statistically significant; while 
other groups spent roughly 20 percent of their 
time (one day in a five-day workweek) on these 
activities, managerial coaches spent 35 percent 
of their time on these activities — or almost two 
days in a five-day workweek. Furthermore, the 
qualitative data (presented below) revealed that 
these coaches viewed their position through 
the lens of their managerial and organizational 
responsibilities. 

Teacher- oriented coaches. Coaches in both the 
fourth and fifth coach categories spent the most 
time working with teachers in a coaching role: 
these activities comprised 41 and 52 percent of the 
workweek for coaches in these groups, respec- 
tively. The two categories differed primarily in the 
mechanism by which this coaching was delivered. 
Coaches in the fourth category spent less time 
working one-on-one with teachers and more time 
working with teachers in a group setting. This was 
significantly greater than the other four categories 
(p < .001). For this reason, we referred to this clus- 
ter as “teacher-oriented (group).” These coaches 
also spent more time working with “out-of-grade” 
teachers (those teaching grades 4-6, the interme- 
diate grades beyond the scope of Reading First). 
About a fifth of coaches (21 percent) were in this 
category. 
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Coaches in the fifth category, in contrast, delivered 
the majority of their coaching to individual teach- 
ers. Again, this was significantly more than the 
other four clusters (p < .001), prompting the name 
“teacher-oriented (individual)” for this group of 
coaches. 

We refer back to table 3 throughout the next 
section, which turns to the qualitative interview 
data to describe each of these coach categories in 
greater detail. 



What do the coach categories look like in practice? 



Because the number of hours a week alone does 
not fully describe the effort and emphasis coaches 
put into their work and because of the inherent 
limitations of self-report data, we turned to the 
coach and teacher interviews from the 77 site 
visits to provide richer descriptions of the differ- 
ent categories of coaches. These interviews did 
not focus on the amount of time spent (except in 
passing reference, such as “spending all my time 
on the data”) but rather on the primary activities 
that coaches undertook and what they felt were the 
major components of their job, as well as teach- 
ers’ views of how their coaches worked with them. 
We combined the coach and teacher interviews to 
create school-level codes indicating the category 
that each coach seemed to fit best. It is important 
to note that even though the majority of coaches 
all do similar activities, both the cluster analysis 
and the qualitative analysis identified distinct 
categories of approaches to performing the job of 
coach. We then looked at the overlap of the survey 
analysis and the qualitative findings to create 
descriptions of each type of coach. 



For data-oriented 
coaches, the collection, 
organization, and use 
of student assessment 
data defined their jobs 



Data-oriented coaches. Coaches 
in this first category were focused 
primarily on student assessment 
data. On average, they spent 45 
percent of their workweek on data 
and assessment-related tasks (see 
table 3). This was more than twice 
as much time as they reported working directly 
with teachers (18 percent). 



For data-oriented coaches, the collection, or- 
ganization, and use of student assessment data 
defined their jobs. One coach described her job in 
this way, “I help people implement the program 
with a higher level of refinement, which means a 
big job is to get data to the people and make sure 
our instructional decisions are linked to the kids 
and their data.” For many data-oriented coaches, 
this work was important because it provided the 
objective information needed to guide a multitude 
of other decisions about how teachers should work 
with students. 

Data-oriented coaches spent, on average, about 13 
percent of their time administering student assess- 
ments. These coaches often took on a great deal 
of the coordination of administration, sometimes 
because they wanted to help teachers and some- 
times because they did not trust others to properly 
administer the tests. One coach noted, “Teacher 
scoring isn’t good enough for them to do the 
benchmark assessments quite yet. We constantly 
re-train and re-direct.” 

Even more than assessment administration, these 
coaches emphasized entering, managing, and 
charting data. They reported spending 16 per- 
cent of their time, on average, on these tasks. (In 
contrast, they reported spending about 14 percent 
of their time providing one-on-one coaching to 
teachers.) They also reported spending 16 percent 
of their time on using and interpreting data. 

Because of their focus on data, these coaches appre- 
ciated professional development that helped them 
in their data-related responsibilities. They particu- 
larly valued sessions such as setting up for assess- 
ment administration, learning to analyze data with 
the principal, and using curriculum-based evi- 
dence. One coach enjoyed the assessment training, 
“especially the analyzing and looking at data in 
several different ways” and said it was “very key for 
us in bumping up our instruction.” 

Not surprisingly, teachers in schools with these 
coaches said that the bulk of their interactions 
with the coach were “focused on assessments.” 
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Another group of teachers noted that their coach 
“keeps us in line, keeps encouraging us to update 
the scores and get students on level .” Sometimes 
teachers said that their coaches were better at 
depicting “where the kids are” than they were 
at helping teachers develop an understanding of 
how to move students forward. In many schools, 
coaches in this category posted enormous wall 
charts showing where each student was located in 
a group or created highly visible hallway displays 
of student progress. Coaches often felt that schools 
were paying attention to the learning of each stu- 
dent in a way they had not in the past. 

Coaches in this group also devoted a substantial 
amount of their time to the use of data— that is, 
interpreting and sharing results and making deci- 
sions based on outcomes. They described spend- 
ing time meeting with teachers and, as a group, 
discussing student assessment outcomes. A coach 
explained: “Having that focus on the data analysis 
is what makes the collaboration happen . . . Every 
other week we have team meetings where I see it 
happening; we look at the data.” 

It was common for coaches to work collaboratively 
with teachers to use data to group students (within 
the classroom or for additional intervention 
outside the reading block). In fact, several of the 
coaches in this category felt that their biggest ac- 
complishments were using data to group students 
effectively for interventions. A coach at a school 
where two coaches shared the position said, “That 
is a strength for us; we spend a lot of time helping 
teachers group students.” At a few schools, coaches 
made grouping decisions on their own. 

Some coaches who fell into the data-oriented 
category were not happy about the amount of time 
that data-related work took. As one coach ex- 
claimed about student data, “There is no end to it!” 
Another coach said the worst part of her job was 
“to get on top of the progress monitoring because 
we had a very large group to assess.” 

At the same time, other data-oriented coaches 
felt that carefully interpreting assessment results 



with teachers was the best way to convince them 
that change was needed and could be effective. 
Some argued that improving assessment results 
was the tool to win over initially skeptical teachers 
and create cohesive support for Reading First in 
their schools. As one coach explained, “In the last 
two years, our teachers have just seen that we are 
moving kids. Every year we are tightening up and 
adjusting our instruction and we have watched the 
number of kids at benchmark and at grade level 
grow. Seeing the success ... It is amazing.” 



Student-oriented 
coaches are 
distinguished by the 
amount of time they 
spend working directly 
with students and the 
comparatively small 
amount of time, in 
contrast, they spend 
working with teachers 



Student- oriented coaches. 

A second category was 
distinguished by the 
amount of time coaches 
spent working directly 
with students and the 
comparatively small 
amount of time, in con- 
trast, they spent working 
with teachers. This was 
even more evident in the 
way they defined their 

work, placing students at the center. Although 
all the coaches had student achievement as the 
ultimate goal of their work, the coaches in this 
cluster were more likely to describe their daily 
tasks by emphasizing their work with students. 
These coaches were more likely to assess students 
directly, to use the results to organize interven- 
tions, and to provide interventions themselves. 
Those three tasks alone took up, on average, about 
a third of these coaches’ workweek (see table 3). 



Like the data-oriented coaches, this group of 
coaches spent a good deal of time regularly collect- 
ing student assessment data (about 11 percent of 
the workweek). One coach explained that, “I enjoy 
doing assessments, seeing where kids are, tracking 
progress, and helping them get the help needed.” 
Coaches in this group used assessment data to 
refine the interventions for students, to recon- 
figure staffing for student interventions, and to 
develop flexible groupings within classrooms— in 
this sense they were quite similar to data-oriented 
coaches. 
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What made this category stand out from the oth- 
ers is these coaches also delivered interventions di- 
rectly to students, spending on average 12 percent 
of their time on this task. One student-oriented 
coach told us, “Because we are a small school, I do 
a lot of the tutoring .” She described working with 
students in the Title I program and substituting 
for absent teachers. In a few instances, coaches 
even taught regularly scheduled portions of the 
reading block. In one school the coach taught the 
third-graders for the first half of the school year 
because the teacher was “difficult to work with” 
and the students were “way behind.” 

In general, this direct work with students ran 
counter to Reading First guidelines, which in 
each of these five states told coaches to work with 
teachers rather than with students. Some coaches 
knew they were pushing the limits of their role, 
while others defended their work with students, 
explaining that it was an indirect way to work with 
teachers. For example, some coaches described 
teaching students as “push-coaching,” a kind of 
gentle modeling of instruction to teachers who did 
not welcome modeling. As one coach described it, 
she and a teacher each took a reading group within 
the same classroom, working at tables alongside 
one another. This provided the teacher a chance to 
see the coach in action, while also giving the coach 
a chance to observe whether the teacher took up 
some of the instructional strategies introduced at 
grade -level meetings. This coach said, “It is amaz- 
ing how much teachers pick up in little lulls when 
you are at their side. You work side-by-side in the 
room, and after school you meet and discuss and 
you modify it. You plan with them continuously. 
You say, c Did this work for you? It didn’t for me. 
Let’s try something different tomorrow.’ We are 
learning side by side.” 

On average, coaches in this category reported 
spending just 10 percent of their time on 
one-on-one coaching and another 4 percent on 
coaching groups of teachers, the least of any of the 
five categories. In addition to the time that work 
with students took up, these coaches were pulled 
away to work on activities unrelated to Reading 



First, such as bus duty, meeting with school 
boards, or substituting for an absent teacher twice 
as often as coaches in other categories. 

Why did these coaches, charged with providing 
professional development to teachers, spend so 
little time working directly with teachers? The an- 
swer was not the same in all cases. Some coaches 
chose to focus on work with students or allowed 
themselves to get pulled into unrelated work 
because they were less comfortable with the role 
of coaching teachers in classrooms. One coach de- 
scribed herself as “an infant” in terms of coaching 
teachers. For others, working with students was a 
way to earn legitimacy in the eyes of the teachers, 
as this coach explained: “I didn’t feel that teachers 
would listen to me unless I demonstrated I was 
one of them. I needed to be with the kids and see 
what was going on.” 

Other coaches clung to the direct work with 
students because it was part of what they had 
always loved about being in schools. These coaches 
said they “needed a dose of kids” and felt “so 
responsible for children”— as one coach explained, 
“That’s the problem with having been a classroom 
teacher.” They cared about students and often had 
deep knowledge of the primary children in their 
school. Some argued that this made them better 
resources to teachers. One coach summed it up 
this way, “I know every kid. When a teacher comes 
to me with a concern about a kid, I know them, 
and we can talk about it.” 

When this group of coaches talked about addi- 
tional training they desired, their wishes were of 
two types. First, they wanted more training on 
specific literacy content, such as vocabulary and 
comprehension, because it supported their direct 
work with students. Second, they wanted more 
training on how to coach teachers, which could 
help them shift some of their focus from students 
to teachers. 

Managerial coaches. Although the coaches in this 
category spent more time working directly with 
teachers than coaches in the first two clusters 
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(about 25 percent of their time), they spent even 
more on managing systems, facilitating meet- 
ings, and keeping up with projects and paperwork 
(about 35 percent of their time; see table 3). This 
type of coach focused on keeping the multiple ele- 
ments of the program running smoothly at their 
schools. In some ways, this group was character- 
ized by coaches who multitasked and spread their 
time more evenly over a wider range of activi- 
ties. When asked about their role, however, these 
coaches highlighted the work they did to meet 
perceived grant expectations for monitoring, orga- 
nizing, and documenting. 

Some of the monitoring and documenting tasks 
involved student data. These coaches were similar 
to student-oriented coaches in the amount of time 
they spent on collecting, managing, and using the 
data on student progress; these tasks took nearly a 
quarter of their workweek. Coaches were mixed on 
whether they felt this data work was a good use of 
their time. Some found it problematic, such as the 
coach who said the worst part of her job was the 
“large amounts of time required for giving DIBELS 
three times a year and analyzing the data.” But for 
others, doing the student assessment was a way of 
supporting Reading First. 

Like coaches in some of the other categories, 
managerial coaches spent a good deal of time 
working with teachers on their instruction — on 
average, 19 percent of their time was spent doing 
one-on-one coaching and another 5 percent work- 
ing with groups of teachers. On top of that work, 
they reported another 18 percent of their time 
went to facilitating and planning meetings, includ- 
ing data-related meetings and study groups. These 
coaches supported teachers by obtaining and shar- 
ing articles summarizing the latest research and 
expectations from the state or finding curriculum 
and classroom materials. One coach said her job 
was “to develop teachers to be the best teachers 
they can be, to be a support and resource to them.” 

Perhaps what most differentiates these coaches 
is the focus on being a “resource” to teachers 
rather than working with them directly. Some 



managerial coaches said they supported teachers 
by obtaining and sharing articles summarizing 
the latest research and expectations from the state 
or finding curriculum and classroom materials. 
Others described finding ways to support teach- 
ers administratively, such as running interference 
for teachers or helping to protect their time. For 
example, a group of teachers in one school said 
that their coach had positively impacted their 
instruction by bringing the interruptions that had 
been occurring during reading time “to a complete 
halt.” 



Just as a subset of student-oriented coaches admit- 
ted working with students because they were 
uncomfortable working directly with teachers, a 
subset of the coaches in the managerial category 
may have sought alternative ways to support 
teachers in order to avoid the one-on-one class- 
room coaching. For example, one coach confessed 
that she “was not very good” at handling any kind 
of resistance from teachers and preferred to find 
other things to do. 



When asked about 
their role, managerial 
coaches highlighted 
the work they did to 
meet perceived grant 
expectations for 
monitoring, organizing, 
and documenting 



Managerial coaches were 
pulled away from their 
jobs to take on other 
duties, such as substitut- 
ing or monitoring the 
cafeteria, although not as 
often as the student-ori- 
ented coaches. One coach 
was asked to leave her 
coaching work to do a lot 
of things, including “making travel arrangements 
for the principal, going to school board meetings 
to get the community involved, providing direct 
interventions, subbing, doing all the progress 
monitoring, and organizing family fun night.” 
These demands tended to be a source of frustra- 
tion for some managerial coaches. 



In fact, a number of coaches in this group were 
not happy about the many tasks that kept them 
busy. Some felt “bogged down” with paperwork 
and found themselves spending a lot of time on 
documentation and keeping “big fat books.” One 
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coach described her position as “being in middle 
management” while another said, “at times it feels 
like I just push papers.” Perhaps because they felt 
overwhelmed with multiple demands, managerial 
coaches appreciated components of professional 
development or state meetings that clarified for 
them in explicit terms what the Reading First pro- 
gram expectations were, such as “how much time 
you need to spend on this, this, and this” or “what 
is coming down the pipe.” 



Despite the commonly 
held image of the 
coach observing and 
conferring with an 
individual teacher, not 
all teacher-oriented 
coaches spent their time 
coaching one-on-one 



Teacher- oriented coaches (group 
and individual). Coaches in the 
fourth and fifth categories are 
most similar to the literatures 
description of reading coaches. 
About a third of the coaches in our 
study fell into one of the teacher- 
oriented categories (group or 
individual), both of which empha- 
sized direct work with teachers to 
develop teacher understanding of literacy develop- 
ment and skill in delivering instruction. 



As shown in table 3, these coaches spent little 
time on data-based tasks, paperwork, or unrelated 
activities, particularly when compared with the 
other types of coaches. They also spent a rela- 
tively small amount of time either planning for or 
directly providing student interventions (a total of 
7-8 percent of their time). 



When teacher-oriented coaches described their 
positions, they focused on teacher professional 
development. Some of the tasks these coaches 
listed as part of their work included showing 
teachers how to implement the core curriculum, 
observing in classrooms and providing feedback, 
demonstrating good teaching, providing in-service 
training and other professional development (such 
as study groups), facilitating teacher meetings, 
helping teachers use data to pinpoint areas for in- 
structional improvement, and acting as a resource 
for both information and materials. 



These coaches described thinking carefully about 
choosing the teachers with whom they would 



work. They invited teachers to come to them if 
they needed help, but they also relied on their 
classroom observations (and those of the prin- 
cipal) to give them insights as to which teachers 
needed assistance. They also used their knowledge 
of students to make decisions: “You go where your 
need is ... I spend more time in ‘strategic 5 class- 
rooms,” explained one coach. 2 Although some of 
these coaches admitted it was easier to work with 
teachers who invited them or who were comfort- 
able working with the coach, they also made an 
effort to work with more challenging teachers. One 
coach said that she kept a check-off sheet and ob- 
servation form to “ensure that I see all teachers . . . 
I try to make time for them when they need me.” 

When working with teachers in classrooms to 
“improve instruction and best practices,” teacher- 
oriented coaches were deeply engaged with the 
work of teaching and learning. As one coach 
explained, they often used the “I do, we do, you 
do,” model, which provided a gradual release of 
responsibility for instruction from the coach to 
the teacher. After an area of instruction had been 
identified for improvement (either by the teacher 
or the coach), the coach began by providing some 
in-service training and then modeling for the 
teacher in the classroom. Occasionally the coach 
arranged for the teacher to see another teacher 
model an aspect of instruction, and the coach and 
teacher observed together and talked about what 
they saw. To get the teacher started, sometimes 
the coach and teacher co-taught. When the coach 
and teacher felt it was time, the teacher took over 
the instruction, with the coach using “whisper 
coaching” and providing feedback. This kind of 
scaffolding ensured that the teacher was ready to 
take over, but many of these coaches continued to 
follow up with regular observations and feedback. 

As a measure of how important working directly 
with teachers was to these coaches, almost all 
mentioned that they would like more professional 
development in this area. They wanted to widen 
their repertoire of skills and deepen their knowl- 
edge of adult learning and support. Several espe- 
cially mentioned wanting to see experts in action. 
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One coach noted that she would like “to go into 
an actual school and have someone demonstrate. 
Let coaches observe in the classroom as someone 
models or gives teachers feedback . . . modeling in 
a real school situation .” 

Group vs. individual coaching. Despite the com- 
monly held image of the coach observing and 
conferring with an individual teacher, not all 
teacher-oriented coaches spent their time coaching 
one-on-one. Coaches in the “individual” category 
spent nearly half their work week (48 percent) 
delivering one-on-one coaching to individual 
teachers and only a small amount of time (3 
percent) working with groups of teachers. These 
coaches also spent dramatically less time work- 
ing with assessment data (11 percent) than any 
other category of coach. In contrast, coaches in the 
“group” category spent about 25 percent of their 
time with individual teachers but used another 10 
percent of their time coaching groups— and spent 
substantially more time (23 percent) working with 
data. The group coaching approach was the more 
common of the two approaches. 

Coaches and teachers reported that group coach- 
ing often occurred during grade-level meetings. 
They described how, prior to the meetings, the 
coach often observed individual teachers. All feed- 
back was generalized to address overall trends, 
and coaches provided that feedback to teachers 
“as a whole group.” One teacher described the way 
coaching worked at her school in this way: “We 
have grade -level meetings at least once a month 
and the reading coach helps us with new ideas and 
feedback about how to improve our teaching.” 

For some coaches a group approach was used 
because it was perceived as “less threatening” 
or to “provide assistance to those teachers who 
refused to be observed in their classes.” For others, 
group coaching was an extension of a collaborative 
culture of professional development in literacy and 
team members supported each other in change. 
One teacher provided an example of how individu- 
als in her school participated in group coaching, 

“I might have a problem with a specific student . . . 



usually what happens is that I bring the challenge 
to a grade-level meeting and we discuss it.” In a 
few schools, coaches described deliberately using 
group coaching as a transition into a form of peer 
coaching, in which teachers “do role playing and 
learn from each other.” Coaches might work with 
a teacher to brainstorm a solution a problem, try 
it out in the classroom, and then together “present 
it at a staff meeting, share it with everyone as a 
refresher.” 



Directive vs. reflective coaching. Another variant 
among teacher-oriented coaches was the tone and 
content of their interactions with teachers. Some 
coaches took a very directive approach, some 
utilized a reflective approach, and many varied 
between reflective and directive depending on the 
circumstance. 



One teacher said of 
her coach, "She can 
be directive if needed. 
She'll just tell us what 
works, but also ask 
what we think . . . she 
can also facilitate 
reflective thinking 
when appropriate." 



Directive coaches acted 
as “experts” and encour- 
aged teachers to teach in 
a particular way. They 
often provided direct 
recommendations, or 
even mandates, about the 
changes that a teacher 
needed to make. Fidel- 
ity of implementation of 
the core reading pro- 
gram was often a primary concern for directive 
coaches. Sometimes teachers asked for specific 
guidance (“How am I supposed to do this?”) 
and the directive coach provided the answer. At 
other times, the coaches initiated the interaction, 
targeting a specific teacher because they noted a 
teacher weakness or because the principal noticed 
a problem and asked the coach to address it. In 
such instances, directive coaches often provided 
explicit direction (or remediation) in an area of 
instruction. 



A teacher explained how it worked in her school: 
“You can ask about something you aren’t sure of 
or she can come in and observe instruction. The 
reading coach will share with teachers what can be 
improved . . . teachers don’t have hours and hours 
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to reflect, but if the reading coach is looking for 
something specific she’ll steer the teacher in the 
right direction.” Although this sometimes created 
conflicts with teachers who disliked being told 
what to do, other teachers appreciated direct and 
explicit messages from the coach. 



No relationship 
existed between 
years of coaching 
experience, teaching 
experience, or advanced 
literacy training with 
coaching category 



Coaches who took a more reflec- 
tive approach attempted to have 
teachers initiate the instruc- 
tional conversation, or asked the 
teachers what they would like to 
work on. One teacher said of her 
coach, “She typically starts with a 
question and will ask something 
like, c What did you think of that 
last lesson? How did it work for you?”’ Many used 
cognitive coaching techniques as an attempt to 
move the teacher forward. In addition to asking 
questions, one coach described working “to draw 
the question out of her so it’s not just me telling 
her. I listened for her to make a comment that I 
could tag off of.” In some cases, reflective coaches 
avoided providing direction in an attempt to mini- 
mize conflict — hence the coach who reportedly 
told teachers, “Everyone is doing great!” But many 
teachers found that this approach helped them 
quickly become comfortable with the coaching 
process. 



In practice, the same coach often used both 
directive and reflective approaches, depending on 
the situation. One teacher said of her coach, “She 
can be directive if needed. She’ll just tell us what 
works, but also ask what we think . . . she can 
also facilitate reflective thinking when appropri- 
ate.” They knew when to provide direct assistance 
and intervention, and they also knew when it was 
important to scaffold the support through reflec- 
tive techniques. These coaches were more direc- 
tive with teachers who were newer or who had 
asked a specific question, and they were more 
reflective with teachers who had more experience 
or who were more resistant. Alternatively, these 
coaches might take different approaches at differ- 
ent times with the same teacher. A teacher noted 
that her coach, “is mostly reflective and allows 



us to come up with our own solutions, but there 
are times that she is directive with me, which I 
appreciate!” 

Coaches in this category tended to care very much 
about the teachers with whom they were working 
and had genuine respect for them, which helped 
them develop effective professional relation- 
ships with the teachers. One coach reported, for 
example, “I take into account the real-life issues 
they have to deal with and their style of teach- 
ing. I help them grow in their own right and not 
a cookie-cutter way. I don’t try to mold them into 
something they are not.” They wanted the teachers 
to “take ownership and become empowered with 
knowledge” and hoped to “empower teachers to 
create sustainability.” While the coaches were very 
focused on teacher growth, they did not forget that 
the ultimate goal was “making changes for kids.” 
Most teachers appreciated this careful attention 
to their work, especially when it was clear that the 
coach valued them as professionals and recognized 
their strengths as well as areas for improvement. 
As one group of teachers claimed, “Because of [the 
coach], we’re better teachers.” 



Relationship of coach categories to other characteristics 

From both the survey and the interview data, dis- 
tinct categories of coaches clearly emerged. Having 
identified five distinctive categories of coaches, 
we next explored three areas that we expected 
might be associated with the category of coach in a 
school: the state in which coaches worked, school 
size, and the educational background and experi- 
ence of coaches. 

Because Reading First project staff at the state level 
organize, plan, and sometimes deliver professional 
development and technical assistance to coaches 
in Reading First schools, we hypothesized that the 
state in which coaches worked might be associ- 
ated with the way in which coaches performed 
their job. Our first analyses therefore looked at the 
relationship between the five states included in the 
study and coach category, using a chi-square test. 
These results are presented in table 4. 
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Analysis revealed that there was a significant rela- 
tionship between state and coach cluster in three 
of five states that we were able to test (chi-square 
p < .01). 3 This is perhaps attributable to the differ- 
ent ways that the states structured their programs 
and the direction given by state-provided profes- 
sional development and technical assistance. 

For example, there were very few data-oriented 
coaches in the only state in which Reading First 
paid for part-time assessment coordinators; in this 
state (Arizona), assessment coordinators took on 
responsibility for administering assessments and 
managing and interpreting data, leaving coaches 
more time to spend in classrooms with teachers. 
Perhaps not surprisingly, this state also had the 
highest proportion of teacher-oriented coaches. 

A variety of state-specific resources, conditions, 
and emphases in training combined to make the 



student- oriented coach category more prevalent in 
Montana, while Washington had a much higher 
percentage of data-oriented coaches than did other 
states. 

We next examined whether school size had a 
relationship to coach category. We suspected 
that coaches at smaller schools, who had fewer 
teachers to coach, might have more time left 
in the day to spend directly with students 
(student-oriented) or on project management 
tasks (managerial). We measured school size in 
two ways: student enrollment and number of 
teachers. Our findings, however, indicated that 
there was no significant relationship between 
coach category and K-3 enrollment (table 5). 
Although the overall relationship between coach 
category and number of teachers was marginally 



TABLE 4 

Distribution of coach categories within each state 



Cluster 1 

Data-oriented 


Cluster 2 

Student- 

oriented 


Cluster 3 

Managerial 


Cluster 4 
Teacher- 
oriented 
(group) 


Cluster 5 
Teacher- 
oriented 
(individual) 


Total 


X 2 


Alaska 


1 


3 


6 


1 


2 


13 


n/a3 


Arizona 


3 


9 


19 


19 


16 


66 


14.91** 


Montana 


5 


17 


5 


6 


0 


33 


12.46** 


Washington 


19 


16 


19 


12 


2 


68 


14.79** 


Wyoming 


0 


0 


6 


3 


1 


10 


n/a3 



* p < .05 ** p < .01 *** p < .001 



TABLE 5 

Mean school size and between group differences by coach category 





Cluster 1 
Data-oriented 


Cluster 2 

Student- 

oriented 


Cluster 3 
Managerial 


Cluster 4 
Teacher- 
oriented 
(group) 


Cluster 5 
Teacher- 
oriented 
(individual) 


ANOVA 
F (4, 173) p 


Mean K-3 


280 


259 


263 


340 


325 


1.59 .178 


enrollment 
















Cluster 1 


Cluster 2 


Cluster 3 


Cluster 4 


Cluster 5 












Teacher- 


Teacher- 








Student- 




oriented 


oriented 


ANOVA 




Data-oriented 


oriented 


Managerial 


(group) 


(individual) 


F (4, 184) p 


Mean 


13.2 


12.1 


11.5 


15.2 


15.4 


2.56 .040 



number of 
teachers 
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significant, post-hoc testing revealed no signifi- 
cant differences between specific categories. This 
continued to hold true even when we converted 
K-3 enrollment into a categorical variable and 
ran a chi-square analysis (data not shown). It 
may be, however, that we were unable to detect a 
relationship to school size because in our sample, 
two-thirds of schools were medium-sized (K-3 
enrollment between 200 and 500). 

Finally, we investigated the possible relationship 
between prior experience and training and how 
coaches spent their time or what they emphasized 



in their work. For example, was someone with 
previous coaching experience more likely to 
emphasize working with teachers? Our analyses 
revealed that this was not the case: no relation- 
ship existed between years of coaching experi- 
ence, teaching experience, or advanced literacy 
training with coaching category. These findings 
are presented in tables 6 and 7. We also grouped 
coaches by their years of previous teaching expe- 
rience (1-3, 4-5, 6-15, and more than 16) and ran 
a chi-square test for differences among categories, 
but again we found no significant relationship 
(data not shown). 



TABLE 6 

Mean coach years of experience and between group differences by coach category 





Cluster 1 
Data-oriented 


Cluster 2 

Student- 

oriented 


Cluster 3 
Managerial 


Cluster 4 
Teacher- 
oriented 
(group) 


Cluster 5 
Teacher- 
oriented 
(individual) 


ANOVA 
F(4, 178) p 


Mean years 
coaching 
experience 
prior to 
Reading First 


1.00 


0.80 


0.55 


0.74 


1.20 


.774 .544 


Mean years 

teaching 

experience 


17.38 


18.69 


15.40 


15.85 


16.75 


.908 .461 


TABLE 7 

Distribution of advanced literacy training by coach category 




Cluster 1 
Data-oriented 


Cluster 2 

Student- 

oriented 


Cluster 3 
Managerial 


Cluster 4 
Teacher- 
oriented 
(group) 


Cluster 5 
Teacher- 
oriented 
(individual) 


Total 3 X 2 


Number with 
advanced 
literacy 
training 


12 


17 


16 


12 


1 


64 


Number 

without 

advanced 

literacy 

training 


16 


25 


31 


21 


10 


103 0.80 



a. The total N for Table 7 is 167, smaller than in the other tables because Arizona coaches did not provide information on their educational background on the 
2006 surveys. For this variable alone, only data from four states were included. 
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NOTES 

1 . 



Note that a few coaches did not complete the 
section of the survey asking about their edu- 
cational background and experience; hence 
the N for this portion of the study was slightly 
lower (184-185 rather than 190 coaches). 

Many coaches in these Reading First schools, 
all of which used the Dynamic Indicators 
of Basic Early Literacy Skills , or DIBELS, to 
assess students, frequently borrowed from 
the DIBELS’ categories of “benchmark,” 



“strategic,” and “intensive” student groups 
to describe teachers and their needs for as- 
sistance. “Strategic” teachers needed spe- 
cific, targeted assistance to help them meet 
expectations. 

3. Two of the states (Alaska and Wyoming) did 
not meet the data requirement of the chi- 
square test that no more than 20 percent of 
the categories should have expected frequen- 
cies of less than 5. Chi-square values are 
therefore not reliable for these states and are 
not reported. 
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APPENDIX 

METHODS 

The purpose of this study is to describe coaching 
in Reading First schools in five western states — 
Alaska, Arizona, Montana, Washington, and 
Wyoming. This includes describing who becomes 
a reading coach, including the background skills 
and qualifications they bring to the position. It 
also includes how they actually perform their jobs; 
specifically, how they spend their time and what 
they see as their focus. To answer these questions, 
this study employs descriptive, mixed methods ap- 
proach, utilizing existing data from the statewide 
evaluations of Reading First in those five states. 

Setting and participants 

Across the five western states in our study, there 
were 203 Reading First grantees in the 2005/06 
school year. Schools were almost uniformly 
high-poverty, with an average of 77 percent of 
students eligible for free- or reduced-price lunch. 
They included both urban (41 percent) and rural 
(28 percent) schools, as well as others in between: 
urban fringe (21 percent) and town (11 percent). 
Almost half of the schools (48 percent) were small, 
serving fewer than 300 students; only a few (9 
percent) were very large, serving more than 500 
students. At least 32 percent of students across 
the 203 schools were considered English language 
learners. 1 



Data sources 
Surveys 

This report relied primarily on data from surveys 
administered to K-3 teachers and, especially, 
literacy coaches at all 203 Reading First schools 
across the five states. A team of evaluators at REL 
Northwest developed the Reading First surveys 
to address the roles, activities, and perceptions 
of reading coaches and teachers. Surveys were 
administered each spring over multiple years 
beginning in 2004; for most of the analyses in 
this report, the 2006 data were used. At times the 



earlier data were employed to determine longitu- 
dinal information, such as whether the coach had 
changed between 2004 and 2006. 

The surveys included more than 200 items de- 
signed to measure attitudes and practices in Read- 
ing First schools. A portion of items measured 
attitudes using a five -point Likert scale while other 
items used ordinal rating scales to measure the 
frequency or intensity of activities. The surveys 
also included a few descriptive checklist items 
about roles and practices. This study drew from 
a series of items that asked coaches to report the 
number of hours they spent on 16 tasks. As with 
most self-report data, coaches’ responses about 
the way they spent their time could have been 
influenced by their beliefs about how they should 
spend it. The confidentiality of the surveys likely 
helped encourage accurate reporting. In addition, 
we noted substantial variation in responses, which 
led us to believe that not all coaches completed the 
series of items based on a preconceived notion of 
what they should do. Although the surveys dif- 
fered slightly from state to state, key items about 
coaching, preparation, and demographics were 
included in all versions. 

Survey response rates were high; in 2006, 97 percent 
of schools returned surveys. In each state, between 
75 and 88 percent of teachers and over 90 percent of 
coaches returned their surveys, providing us with 
complete data from a total of 190 coaches. 

Interviews 

Also in spring 2006, interview data were collected 
from 77 (38 percent) of the Reading First schools 
across the five states. Interviews were conducted 
with 77 reading coaches, 77 principals, and more 
than 300 teachers (one per grade level at each 
school, selected randomly). Interviews at most 
schools (57) were conducted during in-person site 
visits; interviews at some (20) were conducted over 
the telephone. As shown in table Al, the process 
for selecting the schools for qualitative data col- 
lection varied across the five states, depending 
on the needs and preferences of the state project 
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TABLE A1 

Interview sample size and selection method by state 



State 


Number (percentage 
within state) of Reading 
First schools selected 


2006 selection method 


Alaska 


6 (50%) 


Random selection of two schools within each of three districts for site visits. 


Arizona 


27 (37%) 


Remaining schools that had not been visited in previous two years were 
selected for site visits. 


Montana 


16 (48%) 


Half cohort 1 schools randomly selected for telephone interviews. Roughly 
half of cohort 2 schools randomly selected for site visits. 


Washington 


20 (28%) 


Random selection of 20 schools for site visits. 


Wyoming 


10(100%) 


All schools selected for telephone interviews. 



staff. In Alaska, the state asked that each of the 
three participating districts be visited each year; 
within each district, we randomly selected two 
schools to visit. In Arizona, we visited one-third 
of the 73 schools each year for three years. The 
first year, schools were deliberately selected to 
represent a variety of student demographics; in 
the second year, half of the schools that were not 
yet visited were randomly selected; and in the 
third year (spring 2006, the data used for this 
report), the remaining one-third of schools were 
visited. In Montana, half of cohort 1 schools (10 of 
20 schools) were randomly selected for telephone 
interviews, and almost half of cohort 2 schools (6 
out of a total of 13) were randomly selected for site 
visits; telephone interviews utilized the exact same 
protocols as the on-site visits. In Washington, 20 
schools were randomly selected from a pool of 72. 
In Wyoming, telephone interviews were conducted 
with coaches, principals, and teachers at all 10 
schools. 

The different sampling plans in the five states 
mean that the states were differentially repre- 
sented in our qualitative database. We believe that 
this did not undermine the validity of findings, 
however, since the interviews were not used to cal- 
culate frequencies of observed coaching categories, 
but rather to validate and explain the survey data. 

A semi-structured protocol was used to ask 
coaches questions about their roles and respon- 
sibilities as well as the successes and challenges 



of their job; interviews ran for an average of 90 
minutes. Coaches were asked to describe in detail 
how they selected teachers with whom to work, 
how they decided on the content of their work 
with teachers, and how they worked with resistant 
teachers. 

Additionally, interviews with principals (typically 
60 minutes in length) included questions about 
Reading First professional development, technical 
assistance, implementation, and sustainability. 

In focus groups, each lasting about 45 minutes, 
teachers were asked to describe how they worked 
with their reading coach and to provide specific 
examples of their experiences. 

School demographic data 

School demographic information was obtained 
from the U.S. Department of Education National 
Center for Education Statistics (NCES) Com- 
mon Core of Data’s Public Elementary/Secondary 
School Universe Survey, 2003-2004 (vol. la). This 
NCES data set provided basic information and de- 
scriptive statistics on all elementary schools, their 
students, and their teachers. 



Data analyses 

This study used both quantitative analyses of 
the survey data and qualitative analyses of the 
interviews. Combining the two allowed us to bet- 
ter understand how coaches spent their time as 
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well as what they emphasized and valued as they 
performed their jobs. 

Quantitative analyses 

First we conducted basic descriptive analyses on 
variables in the coach surveys from 2006, provid- 
ing us with summary statistics about the educa- 
tion, prior experience, length of workweeks, and 
percentage of time spent on different job responsi- 
bilities. Variation in the last of these, time spent on 
different job responsibilities, led us to take a closer 
look at this variable. 

Cluster analysis was then used to profile coaches 
based upon the percentage of time they devoted 
to the different responsibilities of the job. Clus- 
ter analysis refers to a wide variety of numerical 
procedures that can be used to create a classifica- 
tion scheme for individuals based upon identified 
variables. It uncovers patterns in the underlying 
structure of data and allows for construction of 
groups based upon these patterns. Cluster analysis 
assigns individuals to categories in a manner to 
maximize within-group similarity and between- 
group differences. It does not require knowledge 
of group membership ahead of time, but instead 
derives the classification rule empirically. 

Cluster analysis does not employ a typical statisti- 
cal test of significance, but rather uses algorithms 
that put cases into clusters according to similar- 
ity rules. Cluster analysis methods are used most 
often when researchers are developing a typology 
or classification; this is frequently combined with 
the goal of data exploration (Aldenderfer & Blash- 
field, 1984). Our study utilized cluster analysis for 
these exact purposes: to derive a scheme to classify 
coaches according to how they spent their time 
in schools. This allowed us to explore in greater 
depth our question of how coaches actually spent 
their time on the job, revealing patterns and varia- 
tions across different groups of coaches. 

We used the spring 2006 survey data for the 
cluster analyses. The variables upon which the 
analysis was based were coaches’ self-reports 



on the amount of time they spent on each of 16 
distinct tasks. These tasks were originally identi- 
fied in previous literature on coaching, in qualita- 
tive findings from previous evaluations, as well as 
review of job descriptions and expectations (Costa 
& Garmston, 2002; Walpole & McKenna, 2004; 
Deussen, Autio, Nelsestuen, Roccograndi, & Vale, 
2005; Nelsestuen, Vale, Autio, & Deussen, 2005). 
These 16 survey items were further collapsed into 
12 tasks as shown in table A2. To standardize 
across respondents, whose total working hours 
per week varied, hours spent were converted to 
percentage of time spent on each task per week. 

The cluster analyses for this study were performed 
with the SPSS 13.0 statistical computer package 
(SPSS Inc., 2006) using the two-step method, 
which groups cases into pre-clusters and then ap- 
plies standard hierarchical clustering. The cluster 
solution was validated by significance tests on 
the variables used to create the clusters. One-way 
ANOVAs found statistically significant differ- 
ences across the five coach categories for 11 of the 
12 variables (see table 1 in Findings). A post-hoc 
analysis of each ANOVA was conducted using the 
Scheffe method to determine which categories 
were significantly different from one another and 
the direction of the relationship. 

Finally, we examined the relationships, if any, 
between the coach category and other characteris- 
tics we suspected might be associated with the way 
coaches performed their jobs: the state in which 
they worked, the size of the school, and their own 
educational background. These relationships were 
tested for statistical significance using the chi- 
square or one-way analysis of variance, as appro- 
priate. The nature of the data meant we could not 
assess the direction of the relationships. 

Qualitative analysis 

The qualitative analysis utilized interview data 
from the 77 site visits to further describe and 
understand the emergent coach categories. In 
addition, they helped to counterbalance the 
limitations always present in self-report data by 
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TABLE A2 

Coaching tasks 



Task area 


Variable(s) 




Survey question item(s) 


Coaching 


One-on-one coaching (K— 3) 




Observing, demonstrating or providing feedback to 
individual teachers in grades K-3 




Group coaching (K— 3) 




Training groups of teachers in grades K-3 




Coaching out of grade 




Observing, demonstrating or providing feedback to 
individual teachers in grades 4-6 
Training groups of teachers in grades 4-6 


Assessment 


Administering/coordinating 

assessments 




Coordinating or administering reading assessments 




Managing data 




Managing data (entering data, creating charts, etc.) 




Using/interpreting data 




Reviewing and using reading assessment data 


Interventions 


Planning interventions 




Planning interventions 




Providing interventions directly 




Providing interventions directly to students 


Other 


Attending professional development 




Attending professional development 




Paperwork 




Paperwork 




Planning for/attending meetings 




Planning for meetings 
Attending meetings 




Unrelated (subbing, bus duty, etc.) 




Covering or subbing for teachers 

Bus/recess duty 

Other 



incorporating information from teacher focus 
groups and in-depth interviews with a subset of 
coaches. The initial framework for analysis was in- 
formed by early cluster analysis findings. The data 
were coded to track the types of activities coaches 
engaged in and the different components of their 
job that they emphasized or that teachers and 
principals perceived that they emphasized. A list 
of initial codes related to the clusters was devel- 
oped. Two of the investigators coded a common set 
of interviews to establish reliability and to revise 
the codes to include additional items arising out 
of the data. Inter-rater reliability was 90 percent. 

In the case of discrepancies, the investigators re- 
viewed the transcripts and codes to reach mutual 
agreement. All the 77 site visits were coded using 
the qualitative software program Atlas-ti (Muhr, 
2004). 

After coding, data displays (Miles & Huberman, 
1994) were created to review relationships between 
data types (surveys, interviews, demograph- 
ics). We reviewed interviews from schools that 
exemplified the work of each of the coach types. 



In addition, all interview data were examined for 
discontinuing evidence. Review of these data pro- 
vided further insight into how coaches spent their 
time, what they emphasized and defined as most 
important in their work. The greater level of detail 
provided in the interviews allowed us to flesh out 
the descriptions of what Reading First coaches ac- 
tually did in schools. What we did not do is count 
the number of coaches or teachers who reported 
particular ways of working or certain types of 
challenges. We resisted this for two reasons. First, 
our sample of schools included in the qualitative 
data set was unbalanced across the states (see 
table Al). Second, in some cases we made use of 
information that interviewees volunteered because 
the topic was important to them, rather than 
because they were specifically asked. Therefore 
the information obtained was not always parallel 
across schools. 



Limitations 

There were some important limitations to our 
data on how coaches spent their time that may 
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have influenced our findings. For example, 
coaches were asked to indicate the time they 

spent planning for and attending meetings. Those 

categories did not distinguish between grade- Note 

level meetings, where direct coaching may occur, 
and other unrelated meetings. While the cat- 
egory “coaching groups of teachers” was meant 
to capture coaching at grade-level meetings, it 
was not made explicit in the survey item. Future 
research should distinguish more clearly between 



meetings that contribute to coaching and those 
that do not. 



1 . Percentage of English language learners is 

based upon spring 2006 student demographic 
data from the Dynamic Indicators of Basic 
Early Literacy Skills (DIBELS) database, uti- 
lized at all schools in the study. 
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